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engagement. The BMTN set out to increase the 
number of students deeply engaged in algebra by 
2,019 by the year 2019. Acumulative total of 2,074 
students reported deep engagement in spring 2019, 
exceeding the aim. 


e Teachers continued to learn about student- 
centered instruction. BMTN continued to test and 
refine student-centered instructional routines 
and learned more about specific approaches 
for promoting deep student engagement in the 
math processes of making connections, creating 
justifications, and problem solving. 


e Network-created resources can support spread 
of student-centered instructional routines and 
improvement processes. [he network created 
rubrics, training videos, and a task library to support 
network teachers in learning about and engaging 
in continuous improvement of instruction. These 
resources helped to spread learning to teachers 
within and outside of the network. 
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Network Overview 


The Better Math Teaching Network (BMTN) is a networked improvement community 
(NIC) of researchers, teachers, and instructional leaders who share a common aim: 

to increase the number of students in New England who deeply engage in algebra 
content. We launched the network during the 2016-17 school year and have grown each 
year since. This report describes lessons learned from the 2018-19 school year, our 
third full year as a network. 


Improving Instruction Through Improvement Science 


Fifty-two high school algebra teachers form the core of our improvement community. They develop, test, 
refine, and share instructional routines that are designed to foster deep student engagement in algebra. These 
teachers represent every state in New England and primarily work in urban and rural schools serving high 
proportions of economically disadvantaged students. The network also includes 20 secondary math teachers, 
and two instructional leaders, who participate in BMTN-led improvement communities from two school 
districts in Rhode Island. Their work focuses on learning the tools of improvement science and testing BMTN- 
developed instructional routines in their schools and classrooms. The network's leadership hub consists of five 
members from the American Institutes for Research (AIR) and WestEd. Two researchers with expertise in math 
education and improvement science lead the hub, and they are supported by a math education researcher, a 
project manager, and a communications specialist. 


Our network uses principles from improvement science to structure our work. This approach has been used for 
decades to drive improvements in health care and other industries, but it is somewhat new to education.' The 
approach involves collaborative work using quick-cycle testing to refine instructional routines while paying 
attention to variation in implementation and results. As new learnings are established, promising strategies 
and novel ideas are shared throughout the network, stimulating further improvements and implementation in 
other contexts. To facilitate collaboration and sharing, the BMTN provides the core 52 teachers with structured 
opportunities to meet in-person, as a full group, and virtually, in smaller groups. These structured activities 
total roughly 100 hours per academic year per teacher, though many teachers report spending additional time 
meeting with other teachers informally. Educators in BMTN-led learning communities spend roughly 12 hours 
per year, through four, 2.5-hour meetings at their schools with action periods in between meetings. 


Defining Student-Centered Instruction 


In 2014, BMTN hub leaders and other colleagues published a mixed-methods study of student-centered 

math instruction” The study found a positive association between students’ problem-solving skills and 

the strength of student-centered instruction in their classrooms. The study also produced a framework for 
classifying instructional approaches as strongly student centered. For example, strongly student-centered 
instruction encourages students to communicate their thinking and critique the reasoning of others. We used 
this framework, which built upon prior math education research and expert recommendations,’ to inform initial 
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discussions about the types of instructional improvement work we wanted to carry out as a network. Through 
highly collaborative discussions and ongoing refinements, our network decided to focus on deepening student 
engagement in three mathematical processes: connect, justify, and solve. More specifically: 


e CONNECT. Making connections among mathematical algorithms, concepts, and application to real-world 
contexts, where appropriate. 


e JUSTIFY. Communicating and justifying mathematical thinking as well as critiquing the reasoning of others. 


e SOLVE. Making sense of and solving challenging math problems that extend beyond rote application 
of algorithms. 


Asa network, we test and refine instructional routines that support student engagement within each of these 
processes and share our learnings with each other. We focus the testing primarily on algebra classrooms 
because it is a well-documented “gatekeeper” to advanced math and science high school coursework, and 
subsequent college and STEM-related career opportunities’ 


Network Aim and Driver Diagram 


NICs define and pursue a specific aim to which all improvement efforts are directed. During the first year, we 
created an aim statement that specified the number of students we hoped to deeply engage in the network's 
focal math processes: 


AIM STATEMENT 


By 2019, the BMTN aims to increase the number of New England students who 
connect, justify, and solve with depth by 2,019. 


NICs also construct a working theory of improvement that describes the key levers that are most likely to drive 
improvement toward reaching the aim. A driver diagram is a visual tool used to organize improvement levers 
and associated change ideas to be tested.> The tool also provides network members with a common language 
to guide their collaborative, aligned efforts. The BMTN driver diagram, presented in Exhibit 1, has a primary 
driver of math instruction. The network is focused on changing math instruction so that it provides increased 
opportunities for students to connect, justify, and solve with depth. Teachers can provide these opportunities 
by choosing tasks that support deep engagement and by shifting the academic responsibility from the teacher 
to students. Instructional routines that have these features are defined to be student-centered. We also 
hypothesize that student-centered instructional routines may differ depending on whether the math content 
is new for students. Thus, our secondary driver distinguishes new from previously introduced lesson content. 
The specific change ideas, or instructional routines, that teachers develop, test, refine, and share are aligned 
to both the primary and secondary drivers. These instructional routines are where improvement theory meets 
practice, providing data for continuous improvement to help the network reach its aim. 
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Exhibit 1. BMTN Driver Diagram 


AIM STATEMENT 


Deep Student 
Engagement in Algebra 


2,019 in 2019: 


By 2019, the number of 
students who CONNECT, JUSTIFY, 
and SOLVE with depth in 
algebra will increase by 2,019. 


CONNECT: Make connections 
among mathematical 
algorithms, concepts, and 
application to real-world 
contexts, where appropriate. 


JUSTIFY: Communicate and 
justify mathematical 
thinking as wellas critique 
the reasoning of others. 


SOLVE: Make sense of and 
solve challenging math 
problems that extend 
beyond rote application of 
algorithm. 


Remainder of Report 


q 


PRIMARY DRIVER 
WHAT? 


Mathematics Instruction 


Mathematical instruction 
provides ongoing 
opportunities for all students 
to CONNECT, JUSTIFY, and SOLVE 
in algebra through the choice 
of task/activity and by 
shifting the academic 
responsibility to students. 


Instruction is 
student-centered 


q 


q 


SECONDARY DRIVERS 
WHERE? 


Instructional Routines to 
Introduce New Material 


Instructional Routines 
to Practice/Reinforce 
Previously Introduced 
Material 


CHANGE IDEAS 
HOW? 


The rest of this report is organized by four main sections. In the next section, we describe the process teachers 
used to test and refine their student-centered instructional routines, provide examples of those routines, 

and summarize lessons learned. The following section illustrates how the network supported the spread of 
promising change ideas inside and outside the network and describes key lessons learned. The third section 
presents the data the network collects from teachers and students to track progress towards the aim and 
summarizes our progress to date. The final section identifies next steps for the 2019-20 school year. 
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Testing Student-Centered 
Instructional Routines 


Process for Testing Change Ideas 


One of the key processes used in improvement science is Plan-Do-Study-Act (PDSA) trials. BMTN teachers use 
PDSA trials to test and refine promising change ideas, or instructional routines. First, teachers identify an area 
of focus for their improvement work (connect, justify, or solve) and select a change idea that holds promise for 
improving student engagement in their chosen area of focus. Next, they create a Plan that describes how the 
change idea will be implemented, the supporting data to be collected, and predictions for what the data might 
reveal. Then, in Do, they implement the planned routine, collect data, and note immediate reactions related to 
implementation. In the Study phase, teachers analyze the implementation data more carefully, comparing the 
results with their predictions and beginning to formulate what to try next. During the final phase of the trial, Act, 
teachers decide whether to adopt the current routine, adapt it, or abandon it. Teachers complete several PDSA 
trials individually before meeting with a small group to discuss what they learned with other teachers. At the 
end of the small-group meeting, teachers look across the data collected in each trial and decide what they want 
to test in the next series of trials. The end of the meeting marks the end of one cycle of PDSA testing. The next 
set of trials begins anew PDSA cycle. As illustrated in Exhibit 2, BMTN teachers complete four PDSA cycles per 
year, roughly eight weeks per cycle. Teachers typically complete two or more trials per cycle. 


Exhibit 2: BMTN PDSA Testing Timeframe 


PDSA Cycle 1 PDSA Cycle 2 PDSA Cycle 3 PDSA Cycle 4 
SEPT - OCT NOV - DEC JAN - FEB MAR - APR 
STUDY STUDY STUDY STUDY 


Instructional Routines Tested and Refined in 2018-19 


In previous years, teachers learned the value of using tasks that emphasize math relationships, rather than rote 
procedures; using and removing scaffolds to support deep engagement; providing students with individual 
think time before sharing their thinking with others; and offering opportunities for students to see examples 

of, practice, and receive feedback on their attempt at deep engagement. Teachers also learned that in order 

to emphasize and assess deep student engagement, they needed to have a deep understanding of the math 
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content for themselves and use questions that explicitly address math relationships with their students.” In this 
third year, teachers tested routines that incorporated these elements for each of the three math processes: 
connect, justify, and solve. Overall, about half of the instructional routines that teachers tested this year 
focused on solve and half focused on connect or justify—roughly one-quarter each. 


As shown in Exhibit 3, 12 teachers tested routines designed to support students in making deep connections; 
four of those routines focused on the introduction of new material, seven focused on practicing previously 
learned material, and one focused on both. Each of the routines included individual and small-group work, 
scaffolding, or feedback on the depth of connection. 


Exhibit 3. Instructional Routines Focused on Connect, 2018-19 BMTN Teachers 


Secondary Driver 


Instructional Routine: Connect Intro Practice 


Making mathematical connections through individual and small-group work 


Using mid- and end-of-unit exit tickets with feedback to support deep 
connections 


Connecting big ideas in mathematics to abstract tasks with individual warm-up 
and whole-class discussion 


Using guided questions to support deep connections 


QiCICICI@ 


Using scaffolded writing tasks to promote connections 


Making connections using guided questions and feedback 


Using guided questions, individual and small-group work to support 
connections 


Using exit tickets to determine scaffolding levels for new material and 
connections 


QIal@ 


Incorporating concept maps, individual think time, and small-group work to Y 
make connections to previously learned material 


Using an individual warm up and small-group discussion to facilitate Y 
development of new connections 


Deeper connections using letter writing and video messaging with feedback 


g 
Y 


Using individual, small-group, and whole-group work to facilitate connections XY 


DATA SOURCE: BMTN teacher change idea summaries, 2018-19 school year. 


*“ For more detail on these routines, please see BMTN Lessons Learned reports from the first and second year of implementation - https://www. 
nmefoundation.org/resources/better-math-teaching-network-year-2-resources/ 
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Twelve teachers focused on improving the depth of students’ justifications. Exhibit 4 shows descriptions of 
the routines these teachers tested. All but two of the teachers tested routines that provided opportunities to 
practice previously introduced material. Each of the routines included opportunities for students to reflect 
on their own or others’ justifications through small-group work, whole-group discussion, or by reviewing 
individualized feedback from the teacher. More detail on these routines can also be found on our website 
(www. bettermathteachingnetwork.org). 


Exhibit 4. Instructional Routines Focused on Justify, 2018-19 BMTN Teachers 


Secondary Driver 


Instructional Routine: Justify Intro Practice 
Strengthening justifications through peer feedback XY 
Supporting deep justifications through scaffolded tasks XY 
Supporting deep justifications by facilitating whole-class critique of anonymous Y 
student justifications 

Using exit tickets and time to work with others to promote deep justifications (2 Y 

teachers) 


Supporting justifications by asking peers to present and justify or critique 
others’ solutions 


Incorporating justifications into the individual warm-up and small-group 
discussions 


Q)@ 


Supporting justifications by asking students to pair a student-produced Y 
justification with the teacher feedback 


@ 


Using Google forms with feedback to promote justifications on homework 


Using debating, along with individual think time, to foster deep justifications 


Scaffolding student discourse using claim, evidence, reasoning (CER) writing 


Using individual work followed by classroom presentations to justify and 
compare solution methods 


QC ic 


DATA SOURCE: BMTN teacher change idea summaries, 2018-19 school year. 


Finally, 23 teachers tested routines designed to support students in problem solving with depth. As shown in 
Exhibit 5, five teachers tested routines focused on introducing new material, 11 tested routines focused on 
practicing previously learned material, and six tested routines targeting both. The routines emphasized use of 
scaffolds to support students in each step of the problem-solving process. 
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Exhibit 5. Instructional Routines Focused on Solve, 2018-19 BMTN Teachers 


Secondary Driver 


Instructional Routine: Solve Intro Practice 


Using prompts with structured math talk to facilitate problem solving 


Using problem solving prompts, individual think time, and paired discussions to support 
problem solving 


Qe 


Using prompts anda list of suggestions to use when stuck in the problem-solving 
process 


QI@ 


Annotate; Try Something; Revise; Decide - Scaffolds to support independent solving 


@ 


Using a quiz correction form to promote metacognition 


@ 


Solving non-routine problems in algebra using a solution method template 


Qe 


Scaffolding the process of problem solving 


Using public records and small-group work to foster independence with solving non- 
routine problems 


Using structured student discourse strategies to increase student engagement with 
challenging problems 


@ 


Providing feedback and a template to correct work to promote problem solving with 
depth 


Student choice and whole-class discussion of solutions to warm-up problems 


QIRIQICIQ|@ 


Comparing solution methods to promote deep problem solving (2 teachers) 


C 


Task-based instruction with a problem-solving template to introduce new material 


Analyzing errors to develop problem-solving skills 


Rough draft thinking to support deep problem solving 


Exit tickets with feedback to support problem solving 


@ 
QICI@ 


Using prompts to support independent think time when solving challenging problems 


Assigning roles and providing prompts to support small-group problem solving 


QI@ 


Using common student errors and group corrections to facilitate problem solving 


Using scaffolding to support problem solving 


Using feedback and multiple attempts to support problem solving 


@ 
QIGIQICiC 


Using roles with a template to support problem-solving during group work 


DATA SOURCE: BMTN teacher change idea summaries, 2018-19 school year. 
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More detail on each of the routines presented in this section and those tested in previous years can be found 
on our website (www.bettermathteaching.org). Entries for this year's routines are also accompanied by short 
audio-recordings of teachers describing the routine and what they learned about it through testing. 


Key Learnings 


As teachers shared and discussed what they learned from their testing, one theme for each of the three focal 
math processes emerged: 


e CONNECT. It takes time for students to develop deep connections of abstract concepts. Instructional activities 
should provide opportunities for students to first look for connections in concrete examples before moving 
to more abstract generalizations. 


e JUSTIFY. Students cannot develop deep justifications unless they first have a deep understanding of the 
concepts included in the justification. Instructional activities that ask students to justify their thinking 
should focus on content students understand. 


e SOLVE. Students need support to develop deep problem-solving skills. Instructional activities should use 
problems that can be solved with multiple methods and teachers should offer problem-solving prompts to 
guide students’ problem-solving processes. 


These themes provide more insight into instruction that supports student engagement with connect, justify, 
and solve than had been gleaned in prior years. We elaborate on each in turn. 


Developing deep connections. Our network defines deep connections as the understanding of the relationship 
between two math concepts, a math procedure and the concepts that support that procedure, or two 
procedures. As teachers tested routines to support such depth, they learned that students first needed multiple 
opportunities to make connections with concrete examples. Then, over time, students could begin to generalize 
those connections to more abstract concepts. Exhibit 6 shows an example of a concrete and abstract task that 

a BMTN teacher used to support students in making connections among equations, graphs, and solutions. The 
concrete example requires students to reason about the connections among graphs, equations, and solutions 
using specific equations and a given point: (1,6). In the abstract example, students must generalize their 
understanding of the connections among equations, graphs, and solutions to a situation in which all numbers 
have been removed. 
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Exhibit 6. Sample Task: Moving from Concrete Examples to Abstract Generalizations 


Concrete Example to Support Students in Making Connections Among Equations, Graphs, and Solutions 


Warm Up: Solve by Graphing 


Explain how the graph can help you answer the ix! question. Try to incorporate the bolded Big !deas in your 
explanation. 


Is (1, 6) 4 solution to this system of equations? 


ye4x+2 


yor+6é 


warm up.docx 


Abstract Example to Support Students in Generalizing Connections Among Equations, Graphs, and Solutions 


Connection: The Meaning of Solution in a System 


= Big Ideas: 


¢ Asolution to a system is any set of values that 
makes every equation true. 


¢ Asolution to an equation is any set of values that make the 
equation true. 


In the diagram below, Is point (a, b) a solution to the system represented by the graphs of line g and h? Explain 
your thinking using the Big Ideas above, 


(ab) 


DATA SOURCE: BMTN teacher PDSA testing records, 2018-19 school year. 
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Producing deep justifications. Our network defines deep justifications as those that are clear, relyon math 
relationships and not rote procedures, and have no gaps in reasoning. As teachers tested instructional routines 
designed to support students in producing justifications with these features, they learned that students were 
most successful when the justification activities were presented after, and not during, work to complete a task 
or solve a problem. By justifying work already completed, students were better able to focus on the features 

of a deep justification, including its reliance on math relationships. Exhibit 7 shows a sample task that one 
teacher used to support justifications. The task was presented as a warm-up, addressing previously introduced 
material. They were asked to solve the problem, then write a justification. 


Exhibit 7. Sample Task: Justification Prompt Used in a Warm-Up with Previously Introduced Material 


Nina made two investments. Investment A has a value of $50 at the end of the first year and increases by 
8% per year. Investment B has a value of $60, at the end of the first year and increases by $3 per year. 


Nina checks the value of her investments once a year, at the end of the year. Will investment A’s value 
ever exceed investment B’s value? 


Explain your reasoning and JUSTIFY your answer using complete sentences and appropriate 
mathematical vocabulary. 


DATA SOURCE: BMTN teacher PDSA testing records, 2018-19 school year. 


Engaging in deep problem-solving. Our network defines deep problem solving as students actively making 
sense of what the problem is asking, selecting reasonable solution approaches, monitoring their work and 
changing approaches as needed, and making sense of the final answer. In their testing, teachers learned that 
students more effectively developed deep problem-solving skills when they worked with problems that could 
be solved in multiple ways, enabling them to select an approach that made sense to them. To further support 
students in reasoning about their approaches and solutions in the context of the problem, teachers should 
provide targeted problem-solving prompts at different phases in the problem-solving process. Exhibit 8 shows 
some example problem solving prompts that BMTN teachers used in their testing. 
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Exhibit 8. Sample Prompts: Supporting Deep Problem Solving 


Pre-Thinking Questions 


]. What important information are you given? You can write it below, or underline it in the problem. 


2. In your own words, what is the goal of the task? 


3. Describe what your solution will look like (i.e. Will it be a small number? Willit be a yes orano? Willit bea 
diagram?, etc.). Be as specific as possible! 


4. What are your clarifying questions? (Questions about the provided information. ) 


5. Whatis your initialidea about how you might solve the problem? Check off one method and then explain 
how you plan to use this method. 


Act it Out Draw a Picture Look fora Pattern Guess, Check, & Improve 
Make a List Try a Simpler Case Work Backwards Other: 
Reflection Questions 


6. State your solution. Make sure you responded to the goal of the task. 


7. Explain how you arrived at your solution. What lead you to the answer in question 6? This should bea 
step-by-step explanation of your work. 


8. Explain how you checked your answer. Show work for how you checked your answer. 
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Spreading Instructional Routines and 
Improvement Science Processes 


Another strand of our work focuses on facilitating the spread of promising routines and knowledge of 
improvement science processes. Building on what we have learned in prior years, we continue to implement 
effective strategies and try new approaches to spread knowledge within the network and outside of the 
network through school- and district-based teams. 


Activities Designed to Promote Spread Inside and Outside the Network 


Strategies that have been effective in promoting spread in the past include pairing new teachers with returning 
teachers; encouraging teachers to test change ideas found in Change ![dea Summary books produced by the 
network each year; providing opportunities for teachers to share their learning in whole-network meetings 
throughout the school year; and working with two instructional leaders to support external, school-based 
PLCs focused on testing change ideas developed by network members. This year, we added three new ways 

to promote spread. First, we developed PDSA training videos with interview and classroom footage froma 
BMTN teacher. Second, we finalized common rubrics for assessing depth of engagement to help teachers in 
PDSA testing. Third, we encouraged network teachers to take on leadership roles to spread the work inside and 
outside the network. Each of these new supports facilitated the spread of change ideas and use of the PDSA 
processes to study instructional routines and depth of engagement. 


PDSA training videos. During the second year of the network, we worked with an experienced BMTN teacher 
to develop a series of training videos about how to use PDSA cycles to test and refine instructional routines. 
The videos include a Series of testimonials in which the teacher describes her approach to the Plan, Do, Study, 
and Act phases of a PDSA cycle. The videos also include footage from her classroom to show what happened 
when she implemented the routine and how she analyzed her student work. We used these videos to orient new 
teachers in the core network and to support the two BMTN-led improvement communities in Rhode Island. 
Exhibit 9 describes the organization of a professional learning activity involving the PDSA training videos. 
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Exhibit 9. Professional Learning Activity using PDSA Training Videos 


Activity | Description 
PLAN e Listen to teacher describe the problem she wants to address, her change idea, and how she 
plans to implement the change idea and measure progress 
e Complete the Plan section of the PDSA form 
DO e Watch classroom footage of the teacher implementing the change idea in her classroom. 
e Complete the Do section of the PDSA form 
STUDY e Analyze copies of student work that the teacher collected 
e¢ Complete the Study section of the PDSA form 
e Listen to the teacher's analysis of the student work and identify similarities and differences 
between approaches 
ACT e Complete the Act section of the PDSA form 
e Listen to what the teacher decided to do and identify similarities and differences between the 
conclusions 


Common rubrics. We developed a set of common rubrics, one for each focal math process (connect, justify, 
and solve), to provide teachers with common language and a common way to measure deep engagement in 
algebra. Exhibit 10 shows the rubric for justify. We hypothesized that these common rubrics would support 
spread of ideas among teachers because they would make the analyses and discussion of student work more 
focused and productive, which would better inform ongoing improvements instructional routines. 
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Exhibit 10. Common BMTN Rubric to Evaluate the Depth of Student Justifications 


Justifying Superficially Justifying Deeply 
0 ] 2 3 4 


Logically connected responses 


No Response 


Response is not logical 
or coherent because 
there are several 
missing connections. 


Response is somewhat 
logical and coherent 
with at most a few 
missing connections. 


Response is mostly 
logical and coherent 
but needs some 
organization. 


Response is logical, 
and coherent 
throughout. 


All parts of the 
response are Clearly 
connected to a prior 
step if there is a prior 
step anda following 
stepifthereisa 
following step. 


Clear and precise use of math language (spelling does no 


t matter) 


No Response 


Response has 
inaccurate, missing, 
vague/generic, or 
unnecessary math 
language throughout 
the problem solving/in 
many areas. 


Response has 
inaccurate, missing, 
vague/generic, or 
unnecessary math 
language ina few 
areas. 


Response has 
inaccurate, missing, 
vague/generic, or 
unnecessary math 
language in one area. 


All math language is 

accurately used and 

is appropriate for the 
problem. 


If applicable, all terms, etc. defined and units specified 


No Response 


Response includes 
terms, symbols, 
representations, 
measures but they are 
not defined or specified 


Response includes 
terms, symbols, 
representations, 
measures and some 
are defined or specified 


Response includes 
terms, symbols, 
representations, 
measures and allare 
defined or specified but 
the description is vague 
or unclear 


Response includes 
terms, symbols, 
representations, 
measures and allare 
defined or specified 
and the description is 
clear 


Math relationships (spelling does not matter) 


No Response 


Response does not 
draw on math concepts 
or relationships. 
Instead uses just rules 
or procedures. 


Response contains 

a few math concepts 
or relationships, but 

mostly relies on rules 
or procedures. 


Response contains 
math concepts or 
relationships and rules 
or procedures and is 
evenly split between 
the two. 


Response contains 
more math concepts or 
relationships than rules 
and procedures. 


Teacher leadership. Finally, we offered three leadership opportunities to returning network teachers: leading 
PDSA testing groups, organizing a Task Library for teachers to find and share tasks that hold potential for 
promoting deep engagement in algebra, and supporting improvement work in their schools or districts. Nine 
network teachers participated in these leadership activities and some participated in more than one. Five 
teachers led PDSA testing groups, four teachers led a series of continuous improvement meetings in their 
schools or districts (one teacher was also a PDSA testing group leader), and two teachers led the development 
of the Task Library (one teacher also led continuous improvement meetings in her district). 
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Teachers who led PDSA testing groups engaged in their own improvement projects and then facilitated and 
guided the PDSA testing group to which they were assigned. Hub members did not attend these meetings. To 
become a PDSA testing group leader, teachers needed to have completed at least two years in the network and 
have experience working with teachers in a leadership role (e.g., mentor teacher, work with student teachers, 
instructional coach). Interested teachers completed an application and, once chosen, attended two training 
sessions. The first training session provided an overview of the role and hints for working with teachers. 

We emphasized the difference between being an PSDA facilitator, who focuses on the PDSA process, and an 
instructional coach or mentor, who provides guidance on how to improve instruction. In the second training 
session, PDSA testing group leaders reviewed a completed PDSA form and discussed how they would support 
the teacher in the PDSA process. 


Teachers who organized and led improvement science meetings in their districts and schools did a variety of 
activities. One teacher led several-hour workshops in which she shared her experiences in using PDSA testing 
and encouraged others to do the same. Two teachers led a Series of shorter meetings to support other teachers 
in implementing PDSA testing with a change idea of their choice. Still another teacher led a school-based PLC 
that used PDSA testing to refine strategies for implementing the collaborative learning structures that were 
embedded in their newly adopted math curriculum. We supported each of these teachers by providing guidance 
and materials that we used in our external PLC work. 


Finally, teachers who led the development of the Task Library worked together to create a structure that would 
enable teachers to share, find, and rate tasks that have the potential to support deep engagement with algebra 
content. The Task Library is housed in a Google sheet, with tabs for each of the topics covered in Algebra! (e.g., 
functions, equations). Within each tab, teachers can view a list of tasks, the DEA(s) addressed, the type of task 
(i.e., short task, long task, discovery task), a description of the task, the source, and the teacher who submitted 
it. See Exhibit 11 for an example. Teachers submit tasks by completing a Google form. Once a task is submitted, 
it is reviewed to determine the extent to which it has potential for promoting deep engagement and, if it has 
enough potential, it is added to the library. When teachers use a task, they are encouraged to provide a review 
for others to see. 
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Exhibit 11. Sample Entries in the Task Library 


Resource Primary Secondary Task Type Description Source Submitted by 
DEA DEA 
Cafeteria By Connect Solve Discovery/ Students are introduced to solving multi-step | Inspired by MVP, | Martha 
the Numbers Exploration equations with a problem they are tasked to adopted by Maria 
solve, then guided through understanding Keller 
solving equations. 
Candies Solve None Longer This task helps students think about how | adapted it from | Shawn 
Problem equations can help them to work backwards a puzzle/problem 
through the situation to solve. Students canuse | solving book - 
guess and check or work backwards to start, that | can't find or 


but the goal is that by the end they see how they | remember what 
can create an equation to model the situation. It | it was 

includes pre-thinking and reflection questions 
(second page) to help scaffold the solve process 


for students. 
Cell Phone Solve None Longer This task presents a real-world problem Illustrative Alicia 
Plans requiring the students to write linear equations | Mathematics 


to model different cell phone plans. Looking at 
the graphs of the lines in the context of the cell 
phone plans allows the students to connect the 
meaning of the intersection points of two lines 
with the simultaneous solution of two linear 
equations. 


NOTE: Teacher names have been changed. 


Key Learnings 


As we implemented structures and supports for spreading an understanding of PDSA processes and 
instructional routines to support deep engagement in algebra, we learned the following: 


e Examples of PDSA testing, such as that demonstrated in our training video series, can support teachers in 
learning about and engaging in improvement processes with instructional routines. 


¢ Acommon set of rubrics can support teachers in making improvements that have an impact on student 
engagement and provide a common language for sharing and learning about those improvements. 


e Tools that teachers build, such asa Jask Library, can support continued testing of instructional routines and 
build community among network members. 


We found the videos of the PDSA process to be useful for spreading an understanding of how PDSA testing 
can be used to test and refine instructional routines. Not only did the video activity provide an opportunity for 
new teachers to become familiar with the process, it also provided a means for new and returning teachers to 
build a community with a common understanding of PDSA testing to improve instruction. The videos were also 
well-received by teachers in the external, BMTN-led improvement communities as they provided a model that 
teachers could use in their own testing of change ideas. 
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In contrast to other years, where teachers struggled to develop a meaningful measure of deep engagement, 
about 9 out of ten BMTN teachers used the common BMTN rubrics or close variations of those rubrics in their 
testing. We noted that the conversations in the PDSA testing group and whole-group network meetings were 
more focused with increased attention to the analysis of the evidence collected in each cycle and subsequent 
suggestions for improvement than in previous years. The rubrics appeared to provide a common language 
for discussing the data and associated modifications to the change ideas, which we believe contributed to 
improved data analysis and spread of key learnings relative to student-centered instruction. In addition, 
when used with teachers in the BMTN-led improvement communities, the rubrics offered a vision of deep 
engagement that teachers could draw upon in identifying and subsequently improving change ideas. 


Finally, the Task Library was very well received by the network teachers. Teachers expressed appreciation 
for the resource and looked forward to hearing about new tasks at our network meetings. To date, 89 tasks 
have been reviewed and added to the Jask Library. We anticipate more tasks being added to the library in the 
upcoming year. 


Assessing Progress Towards the Aim 


The network's efforts to foster the testing, refining, and sharing of promising instructional routines are 
intended to help the network reach its aim. As described earlier, the network aim is to increase the number 
of students who deeply engage in connect, justify, and solve by 2,019 by the year 2019. Our primary measure 
for tracking progress towards this aim has been a student survey, which we have administered to students of 
our core network teachers in the fall and spring of each year of the network. We also administer a similarly 
constructed teacher survey to core network teachers to track the frequency with which teachers provide 
students opportunities for deep engagement. We describe and present results from both surveys in this 
section. 


Student Survey 


Our student survey includes 14 core items that ask students to describe the frequency with which they enact 
behaviors that our network has associated with deep engagement in algebra. The survey has five items each 
for connect and justify and four items for solve.” Students select one of the following responses of frequency: 
Never, Rarely, Sometimes, Often, and Almost Daily. To track network progress, we examine overall scores on 
the survey as wellas scores by each of these constructs. Exhibit 12 presents the survey items by construct.” 


* |tems on the survey were informed by the Survey of Chicago Public Schools from the Consortium on Chicago School Research at the University 
of Chicago 


** Reliabilities (Cronbach's Alpha) for the survey: .91 (Allitems), .80 (Connect), .82 (Justify), and .78 (Solve). 
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Exhibit 12. BMTN Student Survey Items by Construct 


Survey Items by Construct and Overall 


CONNECT. How often... 


Do you make sense of mathematical rules, concepts, and relationships? 


Do you make connections to math concepts from other classes you've taken before or in the future? 


Do you make connections between math and real-world situations? 


Do you examine why the steps to solving a math problem or following a procedure work? 


Do you make connections to math concepts you learned previously in this class? 


JUSTIFY. How often... 


Do you explain your answers to others in the class? 


Do you argue or defend your approach to solving math problems? 


Do you critique the mathematical reasoning of others—either written or spoken? 


Do you evaluate other students’ approaches to solving math problems? 


Do you discuss possible solutions to math problems with other students? 


SOLVE. How often... 


Do you keep trying different ways to solve math problems even when they are hard? 


Do you re-read or go over a math problem again if you have trouble understanding it? 


Do you keep working on math problems even when you are stuck? 


Do you determine if your answers to complex math problems make sense? 


DATA SOURCE: BMTN Student Survey, 2018-19 school year. 


To measure deep engagement, we assigned values of 1 to 5 to each item on the survey: Never (1), Rarely (2), 
Sometimes (3), Often (4), and Almost Every Class (5). We do not expect students to be enacting all of these 
behaviors every day, since lessons have different learning goals and structures. Ona test day, for example, 
students do not “explain their answers to others in class.” Similarly, every lesson does not lend itself to 
students “mak[ing] connections to math concepts from other classes or in the future.” Yet, if students Never or 
Rarely enact these behaviors, there is misalignment between the learning environments our network is trying 
to create and what students experience. Thus, we created two evidence levels of deep student engagement 
based on average scores of 3 or higher—i.e., responses of Sometimes or higher. Overall and by construct, 
average scores of 3 or higher provide moderate evidence of deep engagement and average scores of 4 or higher 
provide strong evidence. 
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Exhibit 13 shows the progress the network made towards meeting the aim during the 2018-19 school year. In 
spring 2019, roughly 84% of students (877 total) reported an average of 3 or higher across the 14 core items 
on the survey (moderate evidence of deep engagement). Within the group, 299 students, and 35% overall, 
reported an average of 4 or higher (strong evidence of deep engagement). Solve had the highest percentages 
of strong evidence (50%) and moderate evidence (92%), while justify had the lowest percentages of strong 
(29%) and moderate levels (76%). 


Exhibit 13. Percentage and Number of Students Deeply Engaged in Algebra, by Evidence Level and 
Dimension, 2018-19 School Year 


CONNECT | | = 


76% 
JUSTIFY 


| | | 


| 


SOLVE 


| | 
84%| 877 STUDENTS 


OVERALL 35”| 299 STUDENTS 
0 10 20 30 440 5O 60 70 80 90 100% 
M8 Moderate evidence of deep engagement MB Strong evidence of deep engagement 


DATA SOURCE: BMTN Student Survey, spring 2019; N= 1042. 


Network reaches aim at the moderate level of evidence. Examining student engagement with these measures 
across the three years of the network illustrates overall progress towards the aim. Exhibit 14 shows the 
number of students who reported an average of 3 or higher and 4 or higher overall on the student survey in 
each network year and in total. Some 2,074 students provided at least moderate evidence of deep engagement 
in algebra, which exceeded our aim of 2,019. Among these students, 712 reported strong evidence of deep 


engagement. 
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Exhibit 14. Number of Students Deeply Engaged in Algebra, by Evidence Level and Network Year 


9017 < 
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877 
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MH Moderate evidence of deep engagement MM Strong evidence of deep engagement 


DATA SOURCE. BMTN Student Survey, spring 2017 (N = 447), spring 2018 (N = 997), and spring 2019 (N=1,042). 


Putting BMTN student survey data in a broader context. [hese 14 core student survey items continue to help 
our network track progress towards the aim. But they do not describe how responses from network students 
might compare with samples of students from other schools, districts, and states. For comparison purposes, 
we added five questions to our 2019 survey from the Tripod,’ a widely used student survey that has items that 
are well-aligned to the goals of the BMTN. The Tripod uses a five-point scale to capture the degree to which 
students determine that a given statement is true, ranging from “totally untrue” to “totally true.” Ina prior study 
using the instrument, the top two responses—"mostly true” and “totally true"-were collapsed to indicate student 
agreement with each question.’ Classrooms in which 50 to 80 percent of students responded at these top two 
levels of agreement across the survey were in 75" percentile of the study sample. Classrooms rated at this 
percentile or above had higher levels of student math achievement than classrooms at the 25" percentile and 
below. We only used a subset of Tripod survey items, but at the item level, BMTN students agreed at levels at or 
above classrooms at the 75" percentile in the prior study. Exhibit 15 lists the BMTN student agreement rates—all 
at least 80%-—for each question from the Tripod. 


Exhibit 15. Tripod Survey Items and Levels of BMTN Student Agreement 


Thinking of this class ... Totally or Mostly True 
My teacher accepts nothing more than our full effort. 80% 
My teacher wants me to explain my answers—why | think what | think. 82% 
My teacher doesn't let people give up when the work gets hard. 84% 
My teacher wants us to use our thinking skills, not just memorize things. 84% 
My teacher asks students to explain more about the answers they give. 83% 


DATA SOURCE: BMTN Student Survey, 2018-19 school year, N = 1,042. 
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Teacher Survey 


Using the same core item content from the student survey, we asked teachers to report the frequency that they 
provided students with opportunities for deep engagement. For example, “How often do you make sense of 
mathematical rules, concepts, and relationships?” from the student survey is modified to “How often do you 
provide students with opportunities to make sense of mathematical rules, concepts, and relationships?” on the 
teacher survey. The five response options from the student survey are the same on the teacher survey. We used 
the same method to determine moderate and strong levels of evidence: average scores of 3 for moderate and 
above and 4 and above for strong evidence. 


All but two of BMTN teachers indicated that they provided opportunities for deep student engagement at the 
moderate level, which is perhaps expected given teachers’ active participation in the network and PDSA testing. 
Yet, as shown in Exhibit 16, there was variation among the teachers across these levels. 


Exhibit 16. Frequency Teachers Provided Opportunities for Deep Student Engagement, 
by Evidence Level, 2018-19 


Strong 


SSS Moderate 


1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 
Never Rarely Sometimes Often Almost every class 


DATA SOURCE: BMTN Teacher Survey, 2018-19 school year, N = 52. 


At the strong evidence level, just over half (52%) of BMTN teachers reported at this level of engagement across 
the survey in Spring 2019. For each dimension, teachers reported providing opportunities for connect, justify, 
and solve at 36%, 48%, and 74%, respectively. As shown in Exhibit 17, these percentages are comparable to 
what teachers reported in spring of 2018, by dimension and overall. 
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Exhibit 17. Percentage of Teachers Providing Opportunities for Deep Student Engagement at the Strong 
Evidence Level, Spring of 2018 and 2019 


36” 
CONNECT ae 
48% 
JUSTIFY : 
51% 
74% 
SOLVE on 
52% 
OVERALL rE 
0 10 20 30 40 50 60 70 80% 
M@@8 Spring 2019 $MM Spring 2018 DATA SOURCE: BMTN Teacher Survey, 2018-19 school year, N = 52. 


Exhibit 18 compares the opportunities teachers provided for deep engagement at the strong level with 

what students reported. For each math process and overall, the opportunities teachers provided for deep 
engagement at the strong level were higher than the behaviors reported by students for the corresponding 
survey question. For example, across the survey, 52% of teachers reported providing opportunities for deep 
engagement at the strong level, compared to 35% for students. Connect was the only process in which both the 
opportunities provided and levels of student enactment were almost the same—36% for teachers and 35% for 
students. The math process with the biggest difference was solve: 74% for teachers and 50% for students, a 
difference of 24%. 


Exhibit 18. Percentages of Opportunities for Deep Student Engagement at the Strong Level, Teachers and 
Students, 2018-19 School Year 
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What's Next? 


The network is now entering its fourth and final year. With each year as a network, our understanding of 
student-centered instruction, as we have defined it, has deepened and we expect this to continue in the 
upcoming year. We plan to continue to (1) develop and test new instructional routines that are designed to 
deepen student engagement and improve instruction; (2) spread promising strategies inside and outside the 
network; (3) support teachers as instructional leaders in their schools and districts; and (4) publish findings for 
math education and improvement science audiences. 


More specifically, for spreading our work and building teacher instructional leadership, we plan to devote 
additional resources to support BMTN teachers in lead professional learning communities in their local math 
departments and facilitate a learning community for these leaders. The support will include providing refined 
training materials to teachers, which they can use to guide meetings next year and even beyond. 


We will continue to collect and analyze data that demonstrate progress we are making as a network, as well 
as areas for improvement. In spring 2020, we plan to collect and analyze videos of BMTN teachers’ instruction 
as well as a measure of students’ problem-solving skills. These data sources will be used to complement the 
student and teacher survey data we use to track progress towards the aim. Given how relatively new NICs 

are in education, publishing findings from our work will contribute to the growing evidence for this model of 
collaborative improvement. 
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